Docker Swarm
Orchestrez vos conteneurs
Zikor & π
Et consultez nos formations sur formations.minet.net
Rappels Docker
Un conteneur ?
Une image ?
FROM debian:13
RUN apt-get update && apt-get install cowsay -y
ENV PATH=$PATH:/usr/games
WORKDIR /app
RUN echo "while true; do cowsay < secret.txt; done" > script.sh
RUN chmod +x script.sh
COPY secret.txt ./
CMD ["sh", "/app/script.sh"]
$ docker build -t monimage:latest .
$ docker run -d monimage:latest
5e0f11a974741260b07a3fdeb13add253d758941c2f8000f0850277cce27bacb
$ docker ps
CONTAINER ID IMAGE COMMAND
5e0f11a97474 monimage:latest "sh /app/script.sh"
$ docker logs 5e0f11a97474
(__)\ )\/\
||----w |
|| ||
________________________
< Gustave est magnifique >
------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
$ docker stop 5e0f11a97474
$ docker rm 5e0f11a97474
Un network ?
Permet de lier plusieurs conteneurs entre eux.
- Un réseau virtuel pour lier 2 conteneurs
- Des DNS sont créés automatiquement
$ doker network create mon-network
$ docker run -d --name "brainrot" --network mon-network brainrot-api:latest
40df90101128e2b93b421a770158abd7c8388b9897fb79cf35e05fa9877143a0
$ docker run -it --network mon-network debian:13 bash
root@f8b2e3df984d:/# apt update && apt install curl -y
...
root@f8b2e3df984d:/# curl brainrot:8000
67 67 67 67
Binding de port
$ docker run -d -p "80:8000" brainrot-api:latest
e04c1f248bc8e5f1882cb3e1fcdb55e383ee5000fce50947beee17a119322ee0
$ curl localhost:80
67 67 67 67 67
Rappels Docker Compose
Permet de remplacer les commandes...
bash — mercure@main-vm
$ docker run --network mynetwork --volumes ./assets:/assets --name frontend penpot/backend:latest
$ docker run --network mynetwork -p 80:8000 --volumes ./data:/data --name backend penpot/frontend:latest
$ docker run --network mynetwork --volumes ./data:/var/lib/postgres --name database penpot/database:latest
Permet de remplacer les commandes...
...par un unique fichier
services:
nginx:
image: ${IMAGE_REGISTRY_BASE}/nginx:stable
ports:
- 8001:80
depends_on:
- backend
environment:
- WAIT_FOR_SERVICES=backend:8000 minio:9000
networks:
- calendint_net
backend:
image: ${IMAGE_REGISTRY_BASE}/backend:stable
environment:
- DATABASE_URL=postgresql://postgres:${POSTGRES_PASSWORD}@db:5432/calendint
- APP_BASE_URL=${APP_BASE_URL}
- SECRET_KEY=${SECRET_KEY}
- MINIO_ENDPOINT=minio:9000
- MINIO_ACCESS_KEY=${MINIO_ROOT_USER}
- MINIO_SECRET_KEY=${MINIO_ROOT_PASSWORD}
- MINIO_SECURE=false
- SMTP_HOST=${SMTP_HOST}
- SMTP_PORT=${SMTP_PORT}
- SMTP_USER=${SMTP_USER}
- SMTP_PASSWORD=${SMTP_PASSWORD}
volumes:
- ${VOLUMES}/cal.minet.net/migration_state:/app/migration_state
depends_on:
- db
- minio
networks:
- calendint_net
db:
image: postgres:15
volumes:
- ${VOLUMES}/cal.minet.net/postgres:/var/lib/postgresql/data
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
- POSTGRES_DB=calendint
networks:
- calendint_net
minio:
image: minio/minio:latest
command: server /data --console-address ":9001"
volumes:
- ${VOLUMES}/cal.minet.net/minio:/data
environment:
- MINIO_ROOT_USER=${MINIO_ROOT_USER}
- MINIO_ROOT_PASSWORD=${MINIO_ROOT_PASSWORD}
networks:
- calendint_net
networks:
calendint_net
Lancer un docker compose
bash — mercure@main-vm
$ docker compose up -d
[+] up 5/5
✔ Network calendint_default Created
✔ Container calendint-nginx-1 Started
✔ Container calendint-backend-1 Started
✔ Container calendint-db-1 Started
✔ Container calendint-minio-1 Started
Arreter un docker compose
bash — mercure@main-vm
$ docker compose down
[+] down 5/5
✔ Network calendint_default Removed
✔ Container calendint-nginx-1 Removed
✔ Container calendint-backend-1 Removed
✔ Container calendint-db-1 Removed
✔ Container calendint-minio-1 Removed
Présentation Docker Swarm
Swarm ? Késako ?
- Un outil d'orchestration
- Compatible avec la spec Docker Compose
- Inclus dans Docker
Orchestration ?
Service ?
Un service est une image lancée sur swarm. Elle peut avoir plusieurs réplicas.
Les réplicas, c'est le nombre de conteneurs en load balancing.
Un conteneur est appelé une tâche.
Service ?
Exemple
Le backend de calendint est un service avec 2 réplicas.
Stack ?
Docker Compose permet de créer plusieurs conteneurs ensemble pour former une application.
Docker Swarm permet de créer plusieurs services ensembles pour former une stack.
Avantage
- Lorsqu'un noeud plante, le conteneurs sont migrés sur un autre noeud
- Lorsqu'il y a plusieurs réplicas, les requêtes sont distribuées entre les noeuds
Créer un service
C'est très, très similaire à un fichier Compose !
services:
nginx:
image: nginx/nginx:stable
ports:
- 8001:80
volumes:
- ...
deploy:
replicas: 2
restart_policy:
condition: on-failure
update_config:
parallelism: 2
order: start-first
$ docker stack deploy -c stack.yaml calendint
Les commandes
$ docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
xvt9guw23q4nal0ebjgfeev5m swarm-ln2 Ready Active Leader 29.0.2
a4afmwvbz84caxin6moenw0av swarm-phi Ready Active Reachable 29.0.2
ql5mjc76dcuo2mh2tcoyc9jqf * swarm-pi Ready Active Reachable 29.0.2
$ docker stack ls
NAME SERVICES
calendint 4
$ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
xmrxae60jv4b calendint_backend replicated 2/2 gitlabint.priv.mi...
tdpfa9djhu47 calendint_db replicated 1/1 postgres:15
2y9y7a6got9h calendint_minio replicated 1/1 minio/minio:latest
3paovod53atm calendint_nginx replicated 3/3 gitlabint.priv.mi... *:8001->80/tcp
Les commandes
$ docker stack ps calendint
ID NAME NODE CURRENT STATE
rajhvyf5e4v7 calendint_backend.1 swarm-phi Running 16 hours ago
mbm4h79vol1c \_ calendint_backend.1 swarm-pi Shutdown 3 days ago
wc73vr2296hd calendint_db.1 swarm-phi Running 9 days ago
879sg7wv9ea8 \_ calendint_db.1 swarm-ln2 Shutdown 10 days ago
wgz2mjr7r4iq calendint_minio.1 swarm-pi Running 7 weeks ago
50rnf38ci6g5 \_ calendint_minio.1 swarm-pi Shutdown 7 weeks ago
i4vszhttggef calendint_nginx.1 swarm-ln2 Running 16 hours ago
miyioaci111q \_ calendint_nginx.1 swarm-pi Shutdown 3 days ago
o9ff04gsh0fn calendint_nginx.2 swarm-ln2 Running 16 hours ago
d0v0z3fji95v \_ calendint_nginx.2 swarm-phi Shutdown 3 days ago
Lors d'un déploiement, swarm garde les anciens conteneurs au cas où il faudrait rollback !
Le réseau
Swarm utilise un réseau overlay
Swarm vs Compose
Stockage
- Sur Compose, tout est sur une machine : il n'y a pas de soucis
- Sur swarm, il faut que les noeuds partage un même stockage
- Il ne faut pas utiliser les volumes 'nommés'
Stockage
Plusieurs solutions...
Mot clef: restart
services:
nginx:
restart: on-failure
services:
nginx:
deploy:
restart_policy:
condition: on-failure
Mot clef: depends_on
services:
backend:
depends_on:
db:
condition: service_healthy
database:
healthcheck:
test: [ "CMD-SHELL", "pg_isready -U postgres" ]
interval: 5s
timeout: 5s
retries: 5
services:
backend:
command: "wait-for database:5432 -t 60 ./start.sh"
database:
healthcheck:
test: [ "CMD-SHELL", "pg_isready -U postgres" ]
interval: 5s
timeout: 5s
retries: 5
Mot clef: env_file
services:
backend:
env_file: .env
services:
backend:
environment:
USERNAME: liteapp
SECRET: ${SECRET}
Variable interpolation
Docker remplace les ${VARIABLE} dans les fichiers compose.
services:
example:
image: ${IMAGE}:${VERSION}
$ export IMAGE=hello-world
$ export VERSION=latest
$ docker compose up
example-1 |
example-1 | Hello from Docker!
example-1 | This message shows that your installation appears to be working correctly.
Variable interpolation
La CLI Docker Compose charge automatiquement les variables si un fichier .env est présent.
La CLI de Swarm ne le fait pas : il faut exporter soi-même les variables avant.
$ FOO=bar
$ env | grep FOO
$ export FOO=bar
$ env | grep FOO
FOO=bar
$ set -a
$ source .env
$ set +a
Cluster de Minet
Vue global
CephFS
Dossier /stacks
bash — zikor@swarm-cadance
$ cd $STACKS
$ ls penpot
stack.yaml
CephFS
Dossier /volumes
bash — zikor@swarm-cadance
$ cd $VOLUMES
$ ls
netbox penpot portainer registry calendint
$ ls calendint
minio postgres migration_state
Keepalived
Consultez les IPs des noeuds swarms https://inventaire.minet.net/search/?q=swarm
Reverse Proxy
server {
listen 80;
listen [::]:80;
server_name cal.minet.net;
rewrite ^(.*)$ https://$host$1 permanent;
}
server {
listen 443;
listen [::]:443;
server_name cal.minet.net;
access_log /var/log/nginx/cal.access.log;
error_log /var/log/nginx/cal.error.log;
location / {
proxy_pass http://192.168.102.109:8001/;
}
}
Consultez les configs du Revproxy https://gitlabint.priv.minet.net/minet/configurations/push/revproxy/-/tree/master/sites
Déploiement CICD
En production, on déploie nos stacks depuis une CICD.
Déploiement CICD
spec:
inputs:
folder_name:
default: null
description: "The name of the folder you wish to deploy"
image_registry:
default: null
description: "LEAVE EMPTY ! This variable can only be used by other pipelines"
---
stages:
- prepare
- deploy
variables:
CEPHFS_STACK: "/mnt/cephfs/swarm/stacks/"
FOLDER_NAME: $[[ inputs.folder_name ]]
IMAGE_REGISTRY_BASE: $[[ inputs.image_registry ]]
prepare_deploy:
stage: prepare
tags:
- swarm
variables:
GIT_STRATEGY: none
script:
- cd $CEPHFS_STACK
- git pull origin $CI_COMMIT_REF_NAME
- chmod +x deploy.sh
- if [ ! -z "$FOLDER_NAME" ] && [ ! -z "$IMAGE_REGISTRY_BASE" ]; then echo "$IMAGE_REGISTRY_BASE" > "$FOLDER_NAME/registry-base"; fi
rules:
- if: $CI_COMMIT_BRANCH == "main"
deploy_stack:
stage: deploy
rules:
- if: $CI_PIPELINE_SOURCE == "pipeline"
- if: $CI_PIPELINE_SOURCE == "trigger"
- if: $FOLDER_NAME != ""
variables:
GIT_STRATEGY: none
tags:
- swarm
needs:
- prepare_deploy
script:
- cd $CEPHFS_STACK
- bash ./deploy.sh $FOLDER_NAME
Déploiement CICD
deploy:
stage: deploy
trigger:
project: minet/configurations/push/swarm-prod
branch: main
strategy: depend
inputs:
folder_name: cal.minet.net
image_registry: $CI_REGISTRY_IMAGE
needs:
- build_backend
- build_nginx
Déploiement CICD
#!/usr/bin/env bash
# The user should be gitlab-runner
if [ "$(whoami)" != "gitlab-runner" ]; then
echo "This script must be run as gitlab-runner user"
echo "Example: sudo -u gitlab-runner $0 (stack-path)"
exit 1
fi
if [ -z "$1" ]; then
echo "Usage: $0 (stack-path)"
echo "If the stack-name file is missing, it will be deduced from the stack path"
exit 1
fi
STACK_PATH="/mnt/cephfs/swarm/stacks/$1"
cd "$STACK_PATH" || { echo "The folder $STACK_PATH does not exist."; exit 1; }
# Stack name (everything before the first dot in the folder name)
if [ -f "stack-name" ]; then
STACK_NAME=$(cat stack-name)
else
STACK_NAME=$(basename "$STACK_PATH" | cut -d. -f1)
fi
# IMAGE_REGISTRY_BASE handling (should be created in the prepare stage of the gitlab ci)
if [ -f "registry-base" ]; then
# This will be created by the CI
export IMAGE_REGISTRY_BASE=$(cat registry-base)
fi
if [ -f ".env" ]; then
set -a
. ./.env
set +a
fi
docker stack deploy --with-registry-auth --detach=false -c stack.yaml "$STACK_NAME"